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ABSTRACT 

This study was conducted to determine the effect on 
achievement and attendance of providing students with computer 
assisted instruction (CAI) progress reports. Subjects included 105 
at-risk eleventh graders in a remedial program funded by Holiday Inn. 
In a 5-week period during the summer, subjects were required to 
attend 5 CAI instructional sessions delivered by a WICAT S-300 
computer system. Subjects studied mathematics, language, and reading 
CAI lessons. They were randomly assigned to either the report or the 
no-report group. Reports were provided individually, and the 
no-report group was unaware of the availability of reports. Math 
calculation tests served as the achievement dependent variable. 
Course completion served as the attendance dependent variable. 
Achievement data ANOVA revealed a significant difference between the 
report and no-report groups. Analyses of attendance data revealed 
that providing students with reports increased course completion 
rates across all levels of ability and locus of control. This study 
provides empirical support for the often advised practice of 
providing students w.lth CAI progress reports. (2 references) (GL) 
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Abstract to the educational resources 

INFORMATION CENTER (ERIC)." 

What is the effect on achievement and attendance of providing students with CAI 
progress reports? SubjecUi (Ss) included 105 at-risk eleventh graders in a summer 
remedial program funded by Holiday Inn. Seventeen Ss were omitted from the study due 
to lack of pretest data. In a five week", period during the summer, Ss were required to 
attend 5 CAI instructional sessi ins delivered by a WICAT S-300 computer system. Ss 
studied math, language, and reading CAI lessons. Ss were randomly assigned to either 
the report group or the no-report group. Reports were provided individually, the 
no-report group was un-aware of the availability of reports. Math calculation tests served 
as the achievement dependent variable. Course completion served as the attendance 
dependent variable. 

Achievement data ANOVA revealed a significant difference between the report 
(mean 29.5) and no-report (mean 24.1) groups, p=.004, F= 2.78. Probit (and Logit) 
analyses of attendance data revealed that providing students witii reports increased course 
completion rates across all levels of ability and locus of control. This study provides 
empirical support for the often advised practice of providing students witii CAI progress 
reports. 



A paper presented at the Annual Meeting of the Mid-Soutii Educational Research Association, 
November 8, 1989, in Uttie Rock, Arkansas. 
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Progress Reports Improve Students' Course Completion Rate 
and Achievement in Math Computer-Assisted Instruction 



by Roy B. Clariana and Lana J. Smith 

In computer-assisted instruction (CAI), extensive data on student progress can be 
easily and automatically collected. Nearly all writers advocate the use of progress reports 
in CAI (Alessi and Trollip, 1985), but little research is cited to support this assertion. 

Smith (1988) described the relationship bet^'een different forms of feedback (figure 
1). She argues that feedback contains information that allows the learner to coirect enois. 
Information given about a specific item is microlevel feedback. For example, a teacher 
reviews homework items in class while tiie students check tiieir own paper. Macrolevel 
feedback is feedback tiiat is holistic or general raUier than -.tem specific. For example, 
telling a student that their test score is 86% is macrolevel feedback. Report cards are 
anotiier example of macrolevel feedback. Macrolevel feedback is sometimes called 
advisement and also monitoring feedback (figure 1). 

Feedback studies in CAI usually involve microlevel feedback. The most commonly 
researched microlevel feedback forms are confirmation of response feedback (KOR), 
knowledge of correct response feedback (KCR), and elaborative feedback, KOR or KCR 
feedback may be given immediately, or after an interval of time. Feedback that is not 
given immediately after a student's response is termed delayed feedback, TDelayed 
feedback is anotiier form of microlevel feedback. Well designed CAI should include both 
microlevel and macrolevel feedback are given. This study is concemed witii macrolevel 
feedback, which in tiiis case consists of progress reports presented to students 
individually. 

Learner ability may be an important correlate of course completion. High-ability 
smdents are more successful in academic activities and so may approach CAI 
enrichment with higher motivadon. Low-ability learners may have had littie academic 
success, and CAI enrichment offers one more opportunity for embarrassment. 
High-ability learners may view progress reports as rewarding, since the reports confirm 
that they are doing well. Low-ability leamers may view progress reports as punishing, 
since tiie reports may confirm that tiiey are not domg so well. Alternately, CAI offers 



low-ability students a chance to make leal learning gains that are not norm referenced. 
Anecdotal evidence from commercial software pnxiucers suggests that the atdtudes of 
low-ability students toward academic content improves m CAI environments. CAI 
progress reports, then, may or may not impact the motivation and attitudes of students of 
different abilities. 




...Instnictional treatments that contain no 
feedbxk do not provide infonnation either 
implicitly or explicitly about the accuracy 
ofre^se or explanation. 




...infonns learners of the ccHrect answer 
to the question they have just answered. 




Informational 
Feedback 



■...tells learners, either explicitly or 
in^licitly whether their respraises 
are correct or incocrecL 





.. Jnfonnational, often coirective feedback 
Is infoflnaticn that is provided to learners 
following their response to question within 
aninstoKitio 



...includes those more complex fonns of feedback that 
explain, direct, or monitor. It may or may not be used 
in conjunction with KR and KCR. 



\ 



...includes feedback that e/ plains why 
a cotrect answer is correct andA)r why 
an incorrect answer is incorrect 




...sometimes called advisement, provides 
leamr's with Information as to bow well 
they are performing in the lesson. 




.. jnay cue or prompt learners as to stratifies 
to determine the correct answer. 



Figure 1 . Concept Map of the Types of Infonnation Feedback 



Locus of control (LOG) may also be an important correlate of course completion. 
Learners with a high LOG take responsibility for their actions. Providing GAI progress 
reports to high LOG students should reinforce the idea that hard work pays. Low LOG 
learners feel that they are controlled by circumstance. Providing low LCX^ learners with 
progress reports challenges this assumption. These students may attribute high or low 
scores on activities to external sources and so would disregard the implications of the 
reports. However, the progress repons also represent real and tangible proof of work, or 
lack of work. A low LCX: learner must confront the fact that the sponsoring organization 
is interested in individual progress, and that a day of accounting may be near. Since, low 
LOG respond to external authority, progress reports may increase their rate of course 
completion. GAI progress reports may or may not impact the motivation and attitudes of 
students of different LOG. 

Can locus of control (LOG) and previous achievement (ACT scoies) serve as 
predictors of course completion? Are students with higher achievement more likely to 
complete the course? Are students with an internal locus of control more likely to 
complete the course? 

Our experiences with similar at-risk students showed that attendance and course 
completion would Vie tx problem. In previous years, students have been provided with 
incentives including clothing, mugs, and group vacation trips. Also, weekly follow-up 
phone calls and reaffirming letters have been used. Can progress reports improve course 
completion by these students in this GAI setting? Do progress reports have an effect on 
instruction? Specifically, do ieamers that receive progress reports perform better on math 
post tests than learners that do not receive progress reports? 

Subjects 

The subjects (n=105) in this study were all eleventh grade at-risk students 
participating in the Memphis Partners Senior Prep program sponsored by the Holiday Inn 
Corporation. This program selects students from the Memphis Public School system with 
below average grade points, and ACf scores below 15. Students are recommended for 
the program by school guidance counselors and educators. The Memphis Partners 
program interacts with these students for two years, with the overall goal of improving 
their self concept, academic standing, and college entrance skills. The program goals are 
to increase the probability that th'5 students will receive a college education in onler to 



optimally enter the Memphis economic community. These students arc able to "succeed" 
with just a little help. Seventeen students were omitted from the sample due to lack of 
scorss on either ACT or LOC instruments. This mortality was proportionally divided 
between the control and experimental groups. Tlie final sample contained eighty-eight 
(n=!88) subjects. 

Procedure 

Memphis Partners provides summer job placement, job interview skill training, 
academic instruction, and weekly counselor support with follow-up. Students are 
required to attend all sessions and must also become involved with group community level 
volunteer work such as repairing and painting community buildings. The WICAT CAI 
was part of the academic component and specifically provided language arts and math 
subject content. The CAI reports showed each activity completed, the percent conect, and 
the amount of time taken for each activity. The report also showed the date and exact time 
for starting and finishing each activity. 

Students were required to attend five, three hour sessions. The WICAT lab was 
available five days per week (Monday, Tuesday, Wednesday, Thursday, and Saturday) 
for five weeks from July 5th until August 8th. Two of the five groups were randomly 
selected to receive progress reports (figure 2). 
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Progress Report ProgLCSs Report 

Figure 2. Overview of the Procedure 

Progress reports were provided at the tirst and third sessions (week 1 and week 3) 
about five minutes before the end of the session. This procedure was designed to 
emphasize the fact that the computer was keeping extensive records that would be available 
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to the sponsoring organization. Intentionally, the students wer^ not given time to utilize 
the progress reports for identification of content weakness, if they should be so inclined. 
In this study, reports should effect motivation and attendance, but should not directly 
effect learning performance. Therefore, this study considers the "big brother is watching" 
motivational potential inherent in CAI progress reports. 

Students selected which day they would attend based upon their work schedule. If 
a student missed a scheduled session, they were encouraged to make it up on an alternate 
day. Students attending Tuesday, Thursday, and Saturday did not receive progress 
reports and probably did not know that such records were available. When students 
missed a session, their counselor would phone and encourage the student to make up the 
missed time. Before the program began, and tiien midway tiirough tiie program, the 
associate director sent a letter to each student stating that course completion (attendance at 
five, three hour sessions) was required, and tiiat some penalty would be associated witii 
non-completion. 

At the first sessions, students completed a LOG instrument and a math pretest, as 
well as a standardized reading test. The tests were used to match students to die 
appropriate CAI lesson sequence. 

Experimental Variables 

The dependent variables included math posttest and course completion. The 
variable course completion was assigned 1 for completion and 0 for non-completion. This 
construct was viewed as dichotomous due to the emphasis placed on course completion by 
tiie sponsoring organization. Five was the magic number. Students knew that attendance 
at even four sessions would be viewed as non-completion. The (45) items on tiie matii 
posttest were identical to tiiose on tiie math pretest, and like tiie pretest, each item was 
presented in a totally random order by tiie computer. These items were adapted from 
example problems in matii books from fifdi to twelftii grade levels (tiirough Algebra H). 
The independent variables included tiie American College Test (ACT) scores, math pretest 
scores, and the Nowicki-Strickland Locus of Control (LOC) instrument. 

Due to the dichotomous dependent variable completion, regular analysis is 
unavailable due to violation of tiie assumption of homoscedasticty. In behavioral research, 
dichotomous variables frequently tend to be non-linear. For tiiese reasons, the non-linear 
Probit Regression model was selected. The model equation involved tiie probability of 



course completion given the experimental manipulation of receiving ur not receiving a 
progress report and also the continuous variables ACT and LOG as pt«dictors. 

Results 

Probit and Logit analysis were computed using SPSSx. The Probit analysis proved 
to be a slightly better predictor of course completion. All reported values are derived from 
the Ptobit analysis. 

The model with completion as the dependent variable and report, ACT, and LOG as 
the independent variables showed a goodness-of-fit CHI square of 78.905 with d.f.= 80 
and p=.514 indicating that there was a homogeneous distribution of residuals about the 
regression line, lending support to the model. The parameter estimates for ACT included a 
regression coefficient of 2.350, s.e. of 0.999, with a resulting student t= 2.350 with 
d.f=81 which is just significant at the p=.05 level. The parameter estimates for LOC 
included a regression coefficient of 1.213, s.e. of 0.843, with a resulting student t = 
1.438 with d.f. = 81 which is not significant at the p = .05 level. The intercepts for the 
experimental manipulation report are: with report (1), intercept = 1.649, s.e. = 1.402; and 
with report (0), intercept = 1.193, s.e. = 1.413. For the group report (0) and report (1), 
the probability of completion based on the complete model for student scores in the first, 
second, and third quartile ACT scores are shown in table 1. 



Table 1. Probability of complerion with and witho ut reports at three differ ent 
ACT score ranges (with average LOC for that range) 





JSpOrt 


no report 


ACT 1st quartile 


0.5^ 


0.38 


ACT (median) 


0.61 


0.43 


ACT 3rd quartile 


0.6^ 


0.46 



These results indicate that the group provided with reports were far more likely to 
complete the course than the group not given reports, with an average effect size of 0.33. 

Math pretest achievement data was evaluated for those students who completed the 
required sessions. Comparison of the report and no report groups' math pretest scores 
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(table 2) showed a non-significant difference (table 3)» suggesting that the partial random 
assignment to groups was successful. 

Math posttests for the report and no report groups showed a significant F ratio at 
the p s 0.004 level, suggesting that the treatment condition, progress reports, resulted in a 
significance increase in learning compared to the no report group (effect size: 
(29.5-24, 1)/6.1 = 0.88). These means and F values are summarized in table 2 and 3 
below. The math pretest to posttest effect size for the report group was (29.5-26.2)/6,5 = 
0 51. The math pretest to posttest effect size for the no report group was 24. 1-23.3)75. 1 = 
Qd^. This indicates that the report group gained substantially more from the instruction 
than the no report group. 



Table 2. Math Test Means and Standard Deviations 







Posttest 


Reoort Group r27^ 


26.2 


29,5 




sd6.5 


sd6.2 


No-Report Group 


23.3 


24.1 




sd5.1 


sd6.0 



Table 3, Pre and Posttest ANOVA Summary (M^th\ 



Pretest 




df 


MS 


E 


12 


report or no-report 


98,6 


1 


98,6 


2.78 


0.086 


error 


1629,3 


46 


35,4 






total 


1727.9 


47 








Posttest 




df 


Ma 


£ 


12 


report or no-report 


342.7 


1 


342,7 


2.78 


0.004 


©rror 


1704.6 


46 


37,1 






total 


2047.3 


47 
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Only those students that completed the program had poi'.ttests available for analysis. 
Differential mortality, if it is present, would produce this significant F in two ways, either 
by in- easing the report groups posttest average?; or by decreasing the no report groups 
posttest averages. Increasing the report group's posttest averages would mean that poor 
students in the report condition dropped out. Decreasing the no-report group's posttest 
average would mean that high-ability smdents in the no-report condition dropped out. The 
math pretest scores of the students that dropped out were exammed. These scores were 
unrelated :o treatments, suggesting that differential mortality due to u^atment had not 
occurred. Therefore, progress reports probably improved math achievement. 

Discussion 

Providing students with progi'ess reports appears to have a positive effect on 
students' rate of course completion, and should be recommended for similar groups in 
similar situations. Also, students with higher ACT scores had a significantly higher rate 
of completion. Students with higher extemal LOG scores tended to have a higher course 
completion rate, though this was not shown to be significant with this sample. Contrary 
to expectations, these form of progress report improved math posttest scores, perhaps 
because students with reports worked hiu'der during class. 
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